Skip to content

docs: correct Playwright option support for Browser checks - #821

Merged
hlenke merged 4 commits into
mainfrom
docs/playwright-support-options
Nov 11, 2026
Merged

hlenke merged 4 commits into
mainfrom
docs/playwright-support-options

Conversation

@hlenke

@hlenke hlenke commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

What changed and why

The Playwright support page for Browser checks disagreed with how Browser checks behave:

  • It marked channel as unsupported, while the Browser checks overview lists Google Chrome as a supported browser. channel: 'chrome' works, so the row is now ✅ and a short Use Google Chrome section shows how. The overview links to it.
  • It marked launchOptions as unsupported without saying what happens when you launch a browser yourself. A new Browser launch options section lists the launch options and Chromium arguments chromium.launch() keeps, and the warning logged for dropped arguments.
  • It marked testIdAttribute and deviceScaleFactor (and contextOptions, bypassCSP) as supported, but they only work in the global configuration: test.use() drops them and logs a line. Those rows now say "Global configuration only", and a paragraph lists the options test.use() accepts.
  • The runtime specification's built-in Node.js module list missed dgram, https, os, perf_hooks and stream/promises, and didn't mention the node: prefix. In runtime 2026.04, os and perf_hooks are now marked as partial and the dns settings as applying to one check, matching the sandbox replacements that runtime uses since vm2 3.12.

The edits stay out of the lines #813 changes (video sentence) and don't touch the troubleshooting page (#818).

How each claim was verified

Browser check runners can't run in a local dev stack, so the claims were tested by running real check scripts through the production Browser check runtime for runtime 2026.04 (the vm2 sandbox, the Playwright Test mock and the patched browser launcher from the runner code at monorepo main 84921d2637), with the Playwright config the runner generates for scheduled runs, against real Chromium and Google Chrome.

Claim How it was verified
test.use({ channel: 'chrome' }) runs the check in Google Chrome Run: browser reported Chrome/156… from the Google Chrome binary instead of HeadlessChrome/145…
chromium.launch({ channel: 'chrome' }) also starts Chrome Run: same result for a manual launch
Write channel: 'chrome' literally in the code The runner's own log line asks for a literal string; the scheduler picks a runner with Chrome by matching that literal in the check's files (code)
launchOptions in the global configuration is ignored Run: args from playwrightConfig.use.launchOptions never reached the browser command line; no log
launchOptions in test.use() is ignored and logged Run: Local configuration key "launchOptions" is not supported by Checkly.
chromium.launch() keeps channel, logger, proxy, slowMo, timeout Run: each took effect (Chrome launched, logger called, proxy failed with ERR_PROXY_CONNECTION_FAILED, slowMo: 400 slowed 5 clicks from 0.6 s to 2.2 s, timeout: 1 failed the launch)
Other launch options are ignored; always headless Run: headless: false, devtools: true, executablePath: '/nonexistent/chrome', ignoreDefaultArgs: true had no effect; command line kept --headless
Only --disable-web-security and --host-resolver-rules=... are kept Run: both on the browser command line, a MAP rule resolved a test host; --use-fake-device-for-media-stream, --enable-webgl, --window-size dropped
Warning text for dropped arguments Run: copied from the run log
deviceScaleFactor, testIdAttribute, contextOptions, bypassCSP work in the global configuration Run: devicePixelRatio 2, getByTestId used data-qa, reduced motion on, inline script ran despite a CSP
...and are dropped from test.use() with a log line Run: none applied; one Local configuration key "<key>" is not supported by Checkly. line each
The test.use() allow-list Run: all 20 listed keys passed without a log line and took effect where observable (user agent, viewport, time zone, locale, color scheme, touch); screen, defaultBrowserType, storageState, video, trace, headless, browserName, serviceWorkers were each logged and dropped
Built-in modules incl. dgram, https, os, perf_hooks, stream/promises, and the node: prefix Run: each loaded in a Browser check with and without node:; modules outside the list (e.g.
child_process, net, http2) failed with Cannot find module. The monorepo's runtime harness golden files show the same modules loading in every runtime from 2022.10 to 2026.04

Not tested / left out

  • Not run through a deployed runner or the alternative Browser check runner; that runner carries a copy of the same allow-lists.
  • Left out pending product decisions: setting channel in the global configuration, requesting an exception for a dropped Chromium argument, and listing http as a built-in module (available in Browser checks, not in API check scripts).

🤖 Generated with Claude Code

- Mark channel as supported and explain how to use Google Chrome, which
  the Browser checks overview already lists.
- Explain that launchOptions are ignored, and which launch options and
  Chromium arguments chromium.launch() keeps.
- Mark deviceScaleFactor, testIdAttribute, contextOptions and bypassCSP
  as global configuration only, and list what test.use() accepts.
- Add the missing built-in Node.js modules and the node: prefix to the
  runtime specification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
checkly-422f444a 🟢 Ready View Preview Nov 10, 2026, 1:31 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@hlenke hlenke left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready to merge · The option tables, the test.use() allow-list, the launch-option and Chromium-argument allow-lists, the log lines and the module list all match the Browser check runtime on monorepo main. The inline comments are optional wording.

CI is green and the PR merges cleanly. #813 also edits playwright-support.mdx, but only the video sentence, which this PR does not touch, so either can land first.

Inline comments: 3 wording (all optional).

What I checked
  • Read the whole diff and the full PR versions of playwright-support.mdx and the built-in module section of runtime-specification.mdx (merge base e0e0277f; nothing on main since touches these files).
  • test.use() allow-list (the 20 options listed): identical to the allowed local-configuration keys in the Browser check runtime's Playwright Test mock (packages/check-runtimes-common/src/getLightweightPlaywrightTestMock.js). Any other key is dropped and logs Local configuration key "<key>" is not supported by Checkly., as the PR says. The newer runner logs the same line.
  • launchOptions ignored in the global configuration: the generated Playwright config writes Checkly's own launchOptions over the user's use (getPlaywrightTestConfig.js).
  • chromium.launch() keeps only channel, logger, proxy, slowMo, timeout; always headless; only --disable-web-security and --host-resolver-rules=… args kept; warning text: all match packages/check-runtimes-common/src/browser/getPatchedChromiumProxy.js and its copy in the newer runner. The warning in the PR is the runtime's message verbatim.
  • channel: 'chrome' must be a literal string: the check scheduler picks a runner with Chrome by matching that literal in the check's files, including dependencies and chromium.launch({ channel: 'chrome' }). The runtime logs that the check falls back to Chromium when it runs on a runner without Chrome. That is the basis for the inline suggestion on line 143.
  • Built-in modules (dgram, https, os, perf_hooks, stream/promises, and the node: prefix): match the module allow-list in packages/check-runtimes-common/src/runtimes/runtime-constants.js, which adds every module with and without node:.
  • gh pr checks 821: all pass. gh pr view: MERGEABLE / CLEAN. scripts/pr_impact.py: only an info note that the Browser checks overview is a hub page; the link anchors #use-google-chrome and #browser-launch-options exist on the PR branch.
  • End-to-end: not run against local Checkly. Browser check runners are not part of the local app, so I verified against runtime source. The PR description lists runs through the Browser check runtime.
  • Could not verify: the existing fs (partial…) line, which is not on the PR's lines.

| `trace` | ❌ |
| `video` | ❌ |

In a test file, `test.use()` accepts only these options: `acceptDownloads`, `actionTimeout`, `baseURL`, `channel`, `colorScheme`, `extraHTTPHeaders`, `geolocation`, `hasTouch`, `httpCredentials`, `ignoreHTTPSErrors`, `isMobile`, `javaScriptEnabled`, `locale`, `navigationTimeout`, `offline`, `permissions`, `proxy`, `timezoneId`, `userAgent` and `viewport`. Checkly ignores any other option in `test.use()` and logs a line like this one:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wording (optional): this list includes acceptDownloads and userAgent, but the Use table above has no rows for them. A reader scanning the table would think they're unsupported. Adding two ✅ rows keeps the table and this list in agreement.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add these

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hlenke hlenke left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready to merge · The three commits since my last review (a7709db2..5b5deca4) apply two of my suggestions and add the runtime 2026.04 notes for dns, os and perf_hooks, which match the sandbox replacements on monorepo main. One optional item is left from the last review.

This is an incremental review of a7709db2..5b5deca4. No merge commits in that range. CI is green and the PR merges cleanly. Since the merge base, main changed playwright-support.mdx once, with #813 (the video sentence). GitHub still reports no conflict.

Inline comments: 1 wording (optional).

What I checked
  • Read the full diff a7709db2..5b5deca4 (3 files) and the whole PR version of playwright-support.mdx. Read the built-in module list in runtime-specification.mdx.
  • dns in 2026.04: on monorepo main, the dns replacement keeps the servers and the result order per check sandbox. setServers() and setDefaultResultOrder() never change the runner process resolver (packages/check-runtimes-common/src/vm/sandboxedBuiltins.js). Matches line 50.
  • os in 2026.04: the replacement makes every os function throw os.<name>() is not available in the Checkly check sandbox. It then restores platform(), arch(), type(), endianness(), tmpdir() and homedir(), plus EOL, devNull and constants. That is the exact list on line 53.
  • perf_hooks in 2026.04: the replacement exposes only a per-check performance object. Matches line 55.
  • Only 2026.04: the replacements are enabled only when the runtime version equals 2026.04 (packages/check-runtimes-common/src/doesRuntimeUseVm2v312.js). That backs "in runtime 2026.04" on all three lines.
  • The two applied suggestions (overview Chrome bullet, the literal-string fallback sentence) are the text from my last review.
  • gh pr checks 821: all pass. gh pr view: MERGEABLE / CLEAN. git log e0e0277f..origin/main on the touched files: only #813 (cdb4655a). scripts/pr_impact.py: only the info note that the Browser checks overview is a hub page.
  • End-to-end: not run. Browser check runners are not part of the local app, so I checked these claims against the runtime source (monorepo deb338ba).

@@ -87,20 +87,28 @@ We currently support the following configuration options. We will update this li
| `offline` | ✅ |
| `actionTimeout` | ✅ |
| `navigationTimeout ` | ✅ |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wording (optional): you answered "add these" on my last review, but the Use table still has no acceptDownloads or userAgent row, although the test.use() list below includes both. This adds the two rows:

Suggested change
| `navigationTimeout ` | ✅ |
| `navigationTimeout ` | ✅ |
| `acceptDownloads` | ✅ |
| `userAgent` | ✅ |

@hlenke
hlenke merged commit 9bb1cc1 into main Nov 11, 2026
7 checks passed
@hlenke
hlenke deleted the docs/playwright-support-options branch November 11, 2026 01:57

This branch was successfully deployed

1 active deployment
staging — 5b5deca4 Deployed Nov 10, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant